Load Library Package

“Use the Tidyverse, Luke” – O-W.Kenobi

library(tidyverse)
library(skimr)
library(plotly)

Get Data

Crossref data used from the Setup to the LC OpenRefine Workshop

crossref_data <- read_csv("https://raw.githubusercontent.com/LibraryCarpentry/lc-open-refine/gh-pages/data/doaj-article-sample.csv", 
    col_types = cols(Date = col_date(format = "%m/%d/%Y")))

Take a quick look at the data

glimpse(crossref_data)
Observations: 1,001
Variables: 11
$ Title     <chr> "The Fisher Thermodynamics of Quasi-Probabilities", "Aflatoxin Contamination of the...
$ Authors   <chr> "Flavia Pennini|Angelo Plastino", "Naveed Aslam|Peter C. Wynn", "Rafael R. C. Cuadr...
$ DOI       <chr> "10.3390/e17127853", "10.3390/agriculture5041172", "10.3390/ijms161226101", "10.339...
$ URL       <chr> "https://doaj.org/article/b75e8d5cca3f46cbbd63e91be5b32412", "https://doaj.org/arti...
$ Date      <date> 2015-01-11, 2015-01-11, 2015-01-11, 2015-01-11, 2015-01-11, 2015-01-11, 2015-01-11...
$ Language  <chr> "English", "English", "English", "EN", "EN", "English", "English", "English", "Engl...
$ Subjects  <chr> "Fisher information|quasi-probabilities|complementarity|Physics|QC1-999|Science|Q",...
$ ISSNs     <chr> "1099-4300", "2077-0472", "1422-0067", "2304-6740", "2306-5338", "1420-3049", "2073...
$ Publisher <chr> "MDPI AG", "MDPI AG", "MDPI AG", "MDPI AG", "MDPI AG", "MDPI AG", "MDPI AG", "MDPI ...
$ Citation  <chr> "Entropy, Vol 17, Iss 12, Pp 7848-7858 (2015)", "Agriculture (Basel), Vol 5, Iss 4,...
$ Licence   <chr> "CC BY", "CC BY", "CC BY", "CC BY", "CC BY", "CC BY", "CC BY", "CC BY", "CC BY", "C...
crossref_data

skimr

Skimr is a easy way to have a quick look at the variables in the data frame. In this case the data are mostly character string data. With numeric data skimr will produce a thumbnail histogram (sparkline )

skim(crossref_data)
Skim summary statistics
 n obs: 1001 
 n variables: 11 

-- Variable type:character -----------------------------------------------------
  variable missing complete    n min max empty n_unique
   Authors       0     1001 1001   7 291     0      883
  Citation       0     1001 1001  39 104     0     1000
       DOI      23      978 1001  16  29     0      977
     ISSNs       0     1001 1001   9  19     0       51
  Language      15      986 1001   2   7     0        4
   Licence       6      995 1001   5  11     0        3
 Publisher       0     1001 1001   7  47     0        6
  Subjects       0     1001 1001  17 337     0      988
     Title       0     1001 1001  18 318     0     1000
       URL       0     1001 1001  57  57     0     1000

-- Variable type:Date ----------------------------------------------------------
 variable missing complete    n        min        max     median n_unique
     Date       0     1001 1001 2015-01-01 2015-01-12 2015-01-07       12

Faceting

Two methods to generate a quick table of the languages represented in the dataframe: count() and forcats::fct_count. Since these data are primarily character, it’s helpful to learn about factor data and the forcats package. These two tables are the same. It looks like the data are published in English (spelled two different ways), FRench and Spanish.

crossref_data %>% 
  count(Language)

fct_count(crossref_data$Language, sort = TRUE)

This time, facet on the governing license. All but six articles are covered by a createive commons license.

crossref_data %>% 
  count(Licence)

Facet on the publisher. Sort in descending order.

crossref_data %>% 
  count(Publisher, sort = TRUE)

Facet by authors, and sort by the most prolific. This field appears to be a multi-valued field that is pipe | separated. How do we count and visualize how many articles have multiple authors?

crossref_data %>% 
  count(Authors, sort = TRUE) 

The above table is not very useful (unless tracking publishing teams that are always expressed identically.) Let’s exploring some methods to generate a count of the pipe character separating each author in a single author field. The stringr::str_count() function is a great way to calculate the number of delimiters in each author field.

Note that counting a pipe character | requires using a Regular Expression, or regex. Anyone manipulating string characters with computers will be far more capable after spending some time learning about regular expressions. In this case the we’re looking for a pipe character |. The special trick, here, in understanding regex is to know that a pipe character has special meaning. Therefore we have to escape, or make it know that we want the literal pipe character and not the special meaning pipe character. To escape a character in regex one uses a backslash \. But the weird part is that, in R, one has to escape the the escape character: \\| means look for a literal |.

Below we count the number of pipe characters in each row of the Author field. Using the head function we only display the first six values (rows) in the Author column.

str_count(crossref_data$Authors, "\\|") %>% head()
[1] 1 1 2 3 2 3

Transform Data

Use dplyr::mutate to generate a new field that calculates how many authors each observation contains.

crossref_data %>% 
  select(Authors) %>% 
  mutate(multi_authorship = str_count(Authors, "\\|") + 1) %>% 
  select(Authors, multi_authorship)

Visualize

Authors

Generate a histogram distribution of the multiple authorship variable.

crossref_data %>% 
  select(Authors) %>% 
  mutate(multi_authorship = str_count(Authors, "\\|") + 1) %>% 
  select(multi_authorship, Authors) %>% 
  ggplot() +
  aes(multi_authorship) +
  geom_histogram(binwidth = 1)

This time generate as a bar graph and sort by the most frequent representation. Articles with five authors is the most frequent representation in the dataset.

auth_count <- crossref_data %>% 
  select(Authors) %>% 
  mutate(multi_authorship = str_count(Authors, "\\|") + 1) %>% 
  mutate(multi_authorship = as.character(multi_authorship)) %>% 
  select(multi_authorship, Authors)

ggplot(auth_count) +
  aes(fct_infreq(multi_authorship)) +
  geom_bar() +
  ggtitle("Multiple Authorship")

Explore Subject Headings

Visualize the frequency of multiple subject headings, just as with authors (A bar graph and a histogram)

crossref_data %>% 
  mutate(SH_count = str_count(Subjects, "\\|") + 1) %>% 
  mutate(SH_count = as.character(SH_count)) %>% 
  ggplot() +
  aes(fct_infreq(SH_count)) +
  geom_bar()


crossref_data %>% 
  mutate(SH_count = str_count(Subjects, "\\|") + 1) %>% 
  ggplot() +
  aes(SH_count) +
  geom_histogram(binwidth = 1) 

Data Transformations

Using dplyr, mutate a new variable and transform the data so that ‘EN’ and ‘English’ are the same. Transform ‘ES’ to “Spanish”, and ‘FR’ to “French”.

dplyr::case_when() is one specialized way to perform an if_else transformation.

crossref_data %>% 
  count(Language)

Since EN and English are synonymous, let’s combine them into a single value. case_when is a great function for collapsing values.

crossref_data <- crossref_data %>% 
  mutate(Language = case_when(
    Language == "EN" ~ "English",
    Language == "ES" ~ "Spanish",
    Language == "FR" ~ "French"
  )) 

Visualize the Languages.

Stacked Bar graph shows frequency by Language. Each stack of a bar distinguishes the publishers. English Language is huge and somewhat over-powers the reset of the graph. Make a second graph (below) to drill down on the lesser represented languages.

published_languages_bargraph <- crossref_data %>% 
  ggplot() +
  aes(fct_infreq(Language), fill = Publisher) +
  geom_bar()

published_languages_bargraph

Filter the data to show only the “NA”, “French”, and “Spanish”.

crossref_data %>% 
  filter(is.na(Language) | Language == "French" | Language == "Spanish") %>% 
  ggplot() +
  aes(fct_infreq(Language), fill = Publisher) +
  geom_bar() +
  labs(title = "Published Languages",
       subtitle = "NA or Non-English",
       caption = "Data Source: Crossref.org")

Time Series

published_over_time <- crossref_data %>% 
  count(Date) %>% 
  ggplot(aes(Date, n)) +
  geom_point() +
  geom_line() +
  labs("Publishing Frequency by Day", 
       subtitle = "January, 2015")
  
published_over_time

Interactive

Using Plottly’s ggplotly function, generate visualizations that are available for interactive mousing (i.e. subsetting and exploring). Gadgets such as sliders, drop-down menus, selection boxes and radio buttons are available and especially useful when combining library(crosstalk) with library(flexdashboards) as seen in the opening tab of this demonstration dashboard

ggplotly(published_languages_bargraph)
ggplotly(published_over_time)
LS0tDQp0aXRsZTogImZpcnN0IGxvb2sgYXQgdGhlIGRhdGFzZXQiDQpvdXRwdXQ6DQogIHBkZl9kb2N1bWVudDogZGVmYXVsdA0KICBodG1sX25vdGVib29rOiBkZWZhdWx0DQogIHdvcmRfZG9jdW1lbnQ6IGRlZmF1bHQNCi0tLQ0KDQotIGxpYnJhcnkgZGF0YQ0KLSBhdXRvbWF0aW9uDQoNCiMjIExvYWQgTGlicmFyeSBQYWNrYWdlDQoNCiJVc2UgdGhlIFtUaWR5dmVyc2VdKGh0dHBzOi8vd3d3LnRpZHl2ZXJzZS5vcmcvKSwgTHVrZSIgLS0gTy1XLktlbm9iaSANCg0KYGBge3J9DQpsaWJyYXJ5KHRpZHl2ZXJzZSkNCmxpYnJhcnkoc2tpbXIpDQpsaWJyYXJ5KHBsb3RseSkNCmBgYA0KDQojIyBHZXQgRGF0YQ0KDQpDcm9zc3JlZiBkYXRhIHVzZWQgZnJvbSB0aGUgKipTZXR1cCoqIHRvIHRoZSBbTEMgT3BlblJlZmluZSBXb3Jrc2hvcF0oaHR0cHM6Ly9saWJyYXJ5Y2FycGVudHJ5Lm9yZy9sYy1vcGVuLXJlZmluZS9zZXR1cC5odG1sKQ0KDQpgYGB7cn0NCmNyb3NzcmVmX2RhdGEgPC0gcmVhZF9jc3YoImh0dHBzOi8vcmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbS9MaWJyYXJ5Q2FycGVudHJ5L2xjLW9wZW4tcmVmaW5lL2doLXBhZ2VzL2RhdGEvZG9hai1hcnRpY2xlLXNhbXBsZS5jc3YiLCANCiAgICBjb2xfdHlwZXMgPSBjb2xzKERhdGUgPSBjb2xfZGF0ZShmb3JtYXQgPSAiJW0vJWQvJVkiKSkpDQpgYGANClRha2UgYSBxdWljayBsb29rIGF0IHRoZSBkYXRhDQoNCmBgYHtyfQ0KZ2xpbXBzZShjcm9zc3JlZl9kYXRhKQ0KY3Jvc3NyZWZfZGF0YQ0KYGBgDQoNCiMjIHNraW1yDQoNClNraW1yIGlzIGEgZWFzeSB3YXkgdG8gaGF2ZSBhIHF1aWNrIGxvb2sgYXQgdGhlIHZhcmlhYmxlcyBpbiB0aGUgZGF0YSBmcmFtZS4gIEluIHRoaXMgY2FzZSB0aGUgZGF0YSBhcmUgbW9zdGx5IGNoYXJhY3RlciBzdHJpbmcgZGF0YS4gIFdpdGggbnVtZXJpYyBkYXRhIHNraW1yIHdpbGwgcHJvZHVjZSBhIHRodW1ibmFpbCBoaXN0b2dyYW0gKHNwYXJrbGluZSApDQoNCmBgYHtyfQ0Kc2tpbShjcm9zc3JlZl9kYXRhKQ0KYGBgDQoNCg0KIyMgRmFjZXRpbmcNCg0KVHdvIG1ldGhvZHMgdG8gZ2VuZXJhdGUgYSBxdWljayB0YWJsZSBvZiB0aGUgbGFuZ3VhZ2VzIHJlcHJlc2VudGVkIGluIHRoZSBkYXRhZnJhbWU6ICBgY291bnQoKWAgYW5kIGBmb3JjYXRzOjpmY3RfY291bnRgLiAgU2luY2UgdGhlc2UgZGF0YSBhcmUgcHJpbWFyaWx5IGNoYXJhY3RlciwgaXQncyBoZWxwZnVsIHRvIGxlYXJuIGFib3V0IGZhY3RvciBkYXRhIGFuZCB0aGUgZm9yY2F0cyBwYWNrYWdlLiBUaGVzZSB0d28gdGFibGVzIGFyZSB0aGUgc2FtZS4gIEl0IGxvb2tzIGxpa2UgdGhlIGRhdGEgYXJlIHB1Ymxpc2hlZCBpbiBFbmdsaXNoIChzcGVsbGVkIHR3byBkaWZmZXJlbnQgd2F5cyksIEZSZW5jaCBhbmQgU3BhbmlzaC4NCg0KYGBge3J9DQpjcm9zc3JlZl9kYXRhICU+JSANCiAgY291bnQoTGFuZ3VhZ2UpDQoNCmZjdF9jb3VudChjcm9zc3JlZl9kYXRhJExhbmd1YWdlLCBzb3J0ID0gVFJVRSkNCmBgYA0KDQpUaGlzIHRpbWUsIGZhY2V0IG9uIHRoZSBnb3Zlcm5pbmcgbGljZW5zZS4gIEFsbCBidXQgc2l4IGFydGljbGVzIGFyZSBjb3ZlcmVkIGJ5IGEgW2NyZWF0ZWl2ZSBjb21tb25zXShodHRwczovL2NyZWF0aXZlY29tbW9ucy5vcmcvKSBsaWNlbnNlLg0KDQpgYGB7cn0NCmNyb3NzcmVmX2RhdGEgJT4lIA0KICBjb3VudChMaWNlbmNlKQ0KYGBgDQoNCkZhY2V0IG9uIHRoZSBwdWJsaXNoZXIuICBTb3J0IGluIGRlc2NlbmRpbmcgb3JkZXIuDQoNCmBgYHtyfQ0KY3Jvc3NyZWZfZGF0YSAlPiUgDQogIGNvdW50KFB1Ymxpc2hlciwgc29ydCA9IFRSVUUpDQpgYGANCg0KRmFjZXQgYnkgYXV0aG9ycywgYW5kIHNvcnQgYnkgdGhlIG1vc3QgcHJvbGlmaWMuICBUaGlzIGZpZWxkIGFwcGVhcnMgdG8gYmUgYSBtdWx0aS12YWx1ZWQgZmllbGQgdGhhdCBpcyBwaXBlIGB8YCBzZXBhcmF0ZWQuICBIb3cgZG8gd2UgY291bnQgYW5kIHZpc3VhbGl6ZSBob3cgbWFueSBhcnRpY2xlcyBoYXZlIG11bHRpcGxlIGF1dGhvcnM/DQoNCmBgYHtyfQ0KY3Jvc3NyZWZfZGF0YSAlPiUgDQogIGNvdW50KEF1dGhvcnMsIHNvcnQgPSBUUlVFKSANCmBgYA0KDQpUaGUgYWJvdmUgdGFibGUgaXMgbm90IHZlcnkgdXNlZnVsICh1bmxlc3MgdHJhY2tpbmcgcHVibGlzaGluZyB0ZWFtcyB0aGF0IGFyZSBhbHdheXMgZXhwcmVzc2VkIGlkZW50aWNhbGx5LikgIExldCdzIGV4cGxvcmluZyBzb21lIG1ldGhvZHMgdG8gZ2VuZXJhdGUgYSBjb3VudCBvZiB0aGUgcGlwZSBjaGFyYWN0ZXIgc2VwYXJhdGluZyBlYWNoIGF1dGhvciBpbiBhIHNpbmdsZSBhdXRob3IgZmllbGQuICBUaGUgYHN0cmluZ3I6OnN0cl9jb3VudCgpYCBmdW5jdGlvbiBpcyBhIGdyZWF0IHdheSB0byBjYWxjdWxhdGUgdGhlIG51bWJlciBvZiBkZWxpbWl0ZXJzIGluIGVhY2ggYXV0aG9yIGZpZWxkLiAgDQoNCk5vdGUgdGhhdCBjb3VudGluZyBhIHBpcGUgY2hhcmFjdGVyIGB8YCByZXF1aXJlcyB1c2luZyBhIFJlZ3VsYXIgRXhwcmVzc2lvbiwgb3IgW3JlZ2V4XShodHRwczovL2VuLndpa2lwZWRpYS5vcmcvd2lraS9SZWd1bGFyX2V4cHJlc3Npb24pLiAgQW55b25lIG1hbmlwdWxhdGluZyBzdHJpbmcgY2hhcmFjdGVycyB3aXRoIGNvbXB1dGVycyB3aWxsIGJlIGZhciBtb3JlIGNhcGFibGUgYWZ0ZXIgc3BlbmRpbmcgc29tZSB0aW1lIGxlYXJuaW5nIGFib3V0IHJlZ3VsYXIgZXhwcmVzc2lvbnMuICBJbiB0aGlzIGNhc2UgdGhlIHdlJ3JlIGxvb2tpbmcgZm9yIGEgcGlwZSBjaGFyYWN0ZXIgYHxgLiAgVGhlIHNwZWNpYWwgdHJpY2ssIGhlcmUsIGluIHVuZGVyc3RhbmRpbmcgcmVnZXggaXMgdG8ga25vdyB0aGF0IGEgcGlwZSBjaGFyYWN0ZXIgaGFzIHNwZWNpYWwgbWVhbmluZy4gIFRoZXJlZm9yZSB3ZSBoYXZlIHRvIGVzY2FwZSwgb3IgbWFrZSBpdCBrbm93IHRoYXQgd2Ugd2FudCB0aGUgbGl0ZXJhbCBwaXBlIGNoYXJhY3RlciBhbmQgbm90IHRoZSBzcGVjaWFsIG1lYW5pbmcgcGlwZSBjaGFyYWN0ZXIuICAgVG8gZXNjYXBlIGEgY2hhcmFjdGVyIGluIHJlZ2V4IG9uZSB1c2VzIGEgYmFja3NsYXNoIGBcYC4gIEJ1dCB0aGUgd2VpcmQgcGFydCBpcyB0aGF0LCBpbiBSLCBvbmUgaGFzIHRvIGVzY2FwZSB0aGUgdGhlIGVzY2FwZSBjaGFyYWN0ZXI6ICBgXFx8YCBtZWFucyBsb29rIGZvciBhIGxpdGVyYWwgYHxgLg0KDQpCZWxvdyB3ZSBjb3VudCB0aGUgbnVtYmVyIG9mIHBpcGUgY2hhcmFjdGVycyBpbiBlYWNoIHJvdyBvZiB0aGUgQXV0aG9yIGZpZWxkLiAgVXNpbmcgdGhlIGBoZWFkYCBmdW5jdGlvbiB3ZSBvbmx5IGRpc3BsYXkgdGhlIGZpcnN0IHNpeCB2YWx1ZXMgKHJvd3MpIGluIHRoZSBBdXRob3IgY29sdW1uLg0KDQpgYGB7cn0NCnN0cl9jb3VudChjcm9zc3JlZl9kYXRhJEF1dGhvcnMsICJcXHwiKSAlPiUgaGVhZCgpDQpgYGANCg0KIyMgVHJhbnNmb3JtIERhdGENCg0KVXNlIGBkcGx5cjo6bXV0YXRlYCB0byBnZW5lcmF0ZSBhIG5ldyBmaWVsZCB0aGF0IGNhbGN1bGF0ZXMgaG93IG1hbnkgYXV0aG9ycyBlYWNoIG9ic2VydmF0aW9uIGNvbnRhaW5zLg0KDQpgYGB7cn0NCmNyb3NzcmVmX2RhdGEgJT4lIA0KICBzZWxlY3QoQXV0aG9ycykgJT4lIA0KICBtdXRhdGUobXVsdGlfYXV0aG9yc2hpcCA9IHN0cl9jb3VudChBdXRob3JzLCAiXFx8IikgKyAxKSAlPiUgDQogIHNlbGVjdChBdXRob3JzLCBtdWx0aV9hdXRob3JzaGlwKQ0KYGBgDQoNCiMjIFZpc3VhbGl6ZQ0KDQojIyMgQXV0aG9ycw0KDQpHZW5lcmF0ZSBhIGhpc3RvZ3JhbSBkaXN0cmlidXRpb24gb2YgdGhlIG11bHRpcGxlIGF1dGhvcnNoaXAgdmFyaWFibGUuDQoNCmBgYHtyfQ0KY3Jvc3NyZWZfZGF0YSAlPiUgDQogIHNlbGVjdChBdXRob3JzKSAlPiUgDQogIG11dGF0ZShtdWx0aV9hdXRob3JzaGlwID0gc3RyX2NvdW50KEF1dGhvcnMsICJcXHwiKSArIDEpICU+JSANCiAgc2VsZWN0KG11bHRpX2F1dGhvcnNoaXAsIEF1dGhvcnMpICU+JSANCiAgZ2dwbG90KCkgKw0KICBhZXMobXVsdGlfYXV0aG9yc2hpcCkgKw0KICBnZW9tX2hpc3RvZ3JhbShiaW53aWR0aCA9IDEpDQpgYGANCg0KVGhpcyB0aW1lIGdlbmVyYXRlIGFzIGEgYmFyIGdyYXBoIGFuZCBzb3J0IGJ5IHRoZSBtb3N0IGZyZXF1ZW50IHJlcHJlc2VudGF0aW9uLiAgQXJ0aWNsZXMgd2l0aCBmaXZlIGF1dGhvcnMgaXMgdGhlIG1vc3QgZnJlcXVlbnQgcmVwcmVzZW50YXRpb24gaW4gdGhlIGRhdGFzZXQuIA0KDQpgYGB7cn0NCmF1dGhfY291bnQgPC0gY3Jvc3NyZWZfZGF0YSAlPiUgDQogIHNlbGVjdChBdXRob3JzKSAlPiUgDQogIG11dGF0ZShtdWx0aV9hdXRob3JzaGlwID0gc3RyX2NvdW50KEF1dGhvcnMsICJcXHwiKSArIDEpICU+JSANCiAgbXV0YXRlKG11bHRpX2F1dGhvcnNoaXAgPSBhcy5jaGFyYWN0ZXIobXVsdGlfYXV0aG9yc2hpcCkpICU+JSANCiAgc2VsZWN0KG11bHRpX2F1dGhvcnNoaXAsIEF1dGhvcnMpDQoNCmdncGxvdChhdXRoX2NvdW50KSArDQogIGFlcyhmY3RfaW5mcmVxKG11bHRpX2F1dGhvcnNoaXApKSArDQogIGdlb21fYmFyKCkgKw0KICBnZ3RpdGxlKCJNdWx0aXBsZSBBdXRob3JzaGlwIikNCmBgYA0KDQojIyMgRXhwbG9yZSBTdWJqZWN0IEhlYWRpbmdzDQoNClZpc3VhbGl6ZSB0aGUgZnJlcXVlbmN5IG9mIG11bHRpcGxlIHN1YmplY3QgaGVhZGluZ3MsIGp1c3QgYXMgd2l0aCBhdXRob3JzIChBIGJhciBncmFwaCBhbmQgYSBoaXN0b2dyYW0pDQoNCmBgYHtyfQ0KY3Jvc3NyZWZfZGF0YSAlPiUgDQogIG11dGF0ZShTSF9jb3VudCA9IHN0cl9jb3VudChTdWJqZWN0cywgIlxcfCIpICsgMSkgJT4lIA0KICBtdXRhdGUoU0hfY291bnQgPSBhcy5jaGFyYWN0ZXIoU0hfY291bnQpKSAlPiUgDQogIGdncGxvdCgpICsNCiAgYWVzKGZjdF9pbmZyZXEoU0hfY291bnQpKSArDQogIGdlb21fYmFyKCkNCg0KY3Jvc3NyZWZfZGF0YSAlPiUgDQogIG11dGF0ZShTSF9jb3VudCA9IHN0cl9jb3VudChTdWJqZWN0cywgIlxcfCIpICsgMSkgJT4lIA0KICBnZ3Bsb3QoKSArDQogIGFlcyhTSF9jb3VudCkgKw0KICBnZW9tX2hpc3RvZ3JhbShiaW53aWR0aCA9IDEpIA0KYGBgDQoNCg0KIyMgRGF0YSBUcmFuc2Zvcm1hdGlvbnMNCg0KVXNpbmcgZHBseXIsIG11dGF0ZSBhIG5ldyB2YXJpYWJsZSBhbmQgdHJhbnNmb3JtIHRoZSBkYXRhIHNvIHRoYXQgJ0VOJyBhbmQgJ0VuZ2xpc2gnIGFyZSB0aGUgc2FtZS4gIFRyYW5zZm9ybSAnRVMnIHRvICJTcGFuaXNoIiwgYW5kICdGUicgdG8gIkZyZW5jaCIuICANCg0KYGRwbHlyOjpjYXNlX3doZW4oKWAgaXMgb25lIHNwZWNpYWxpemVkIHdheSB0byBwZXJmb3JtIGFuIGBpZl9lbHNlYCB0cmFuc2Zvcm1hdGlvbi4NCg0KYGBge3J9DQpjcm9zc3JlZl9kYXRhICU+JSANCiAgY291bnQoTGFuZ3VhZ2UpDQpgYGANCg0KU2luY2UgYEVOYCBhbmQgYEVuZ2xpc2hgIGFyZSBzeW5vbnltb3VzLCBsZXQncyBjb21iaW5lIHRoZW0gaW50byBhIHNpbmdsZSB2YWx1ZS4gIGBjYXNlX3doZW5gIGlzIGEgZ3JlYXQgZnVuY3Rpb24gZm9yIGNvbGxhcHNpbmcgdmFsdWVzLg0KDQpgYGB7cn0NCmNyb3NzcmVmX2RhdGEgPC0gY3Jvc3NyZWZfZGF0YSAlPiUgDQogIG11dGF0ZShMYW5ndWFnZSA9IGNhc2Vfd2hlbigNCiAgICBMYW5ndWFnZSA9PSAiRU4iIH4gIkVuZ2xpc2giLA0KICAgIExhbmd1YWdlID09ICJFUyIgfiAiU3BhbmlzaCIsDQogICAgTGFuZ3VhZ2UgPT0gIkZSIiB+ICJGcmVuY2giDQogICkpIA0KYGBgDQoNCiMjIyBWaXN1YWxpemUgdGhlIExhbmd1YWdlcy4gDQoNClN0YWNrZWQgQmFyIGdyYXBoIHNob3dzIGZyZXF1ZW5jeSBieSBMYW5ndWFnZS4gRWFjaCBzdGFjayBvZiBhIGJhciBkaXN0aW5ndWlzaGVzIHRoZSBwdWJsaXNoZXJzLiBFbmdsaXNoIExhbmd1YWdlIGlzIGh1Z2UgYW5kIHNvbWV3aGF0IG92ZXItcG93ZXJzIHRoZSByZXNldCBvZiB0aGUgZ3JhcGguICBNYWtlIGEgc2Vjb25kIGdyYXBoIChiZWxvdykgdG8gZHJpbGwgZG93biBvbiB0aGUgbGVzc2VyIHJlcHJlc2VudGVkIGxhbmd1YWdlcy4NCg0KYGBge3J9DQpwdWJsaXNoZWRfbGFuZ3VhZ2VzX2JhcmdyYXBoIDwtIGNyb3NzcmVmX2RhdGEgJT4lIA0KICBnZ3Bsb3QoKSArDQogIGFlcyhmY3RfaW5mcmVxKExhbmd1YWdlKSwgZmlsbCA9IFB1Ymxpc2hlcikgKw0KICBnZW9tX2JhcigpDQoNCnB1Ymxpc2hlZF9sYW5ndWFnZXNfYmFyZ3JhcGgNCmBgYA0KDQpGaWx0ZXIgdGhlIGRhdGEgdG8gc2hvdyBvbmx5IHRoZSAiTkEiLCAiRnJlbmNoIiwgYW5kICJTcGFuaXNoIi4NCg0KYGBge3J9DQpjcm9zc3JlZl9kYXRhICU+JSANCiAgZmlsdGVyKGlzLm5hKExhbmd1YWdlKSB8IExhbmd1YWdlID09ICJGcmVuY2giIHwgTGFuZ3VhZ2UgPT0gIlNwYW5pc2giKSAlPiUgDQogIGdncGxvdCgpICsNCiAgYWVzKGZjdF9pbmZyZXEoTGFuZ3VhZ2UpLCBmaWxsID0gUHVibGlzaGVyKSArDQogIGdlb21fYmFyKCkgKw0KICBsYWJzKHRpdGxlID0gIlB1Ymxpc2hlZCBMYW5ndWFnZXMiLA0KICAgICAgIHN1YnRpdGxlID0gIk5BIG9yIE5vbi1FbmdsaXNoIiwNCiAgICAgICBjYXB0aW9uID0gIkRhdGEgU291cmNlOiBDcm9zc3JlZi5vcmciKQ0KYGBgDQoNCiMjIFRpbWUgU2VyaWVzDQoNCmBgYHtyfQ0KcHVibGlzaGVkX292ZXJfdGltZSA8LSBjcm9zc3JlZl9kYXRhICU+JSANCiAgY291bnQoRGF0ZSkgJT4lIA0KICBnZ3Bsb3QoYWVzKERhdGUsIG4pKSArDQogIGdlb21fcG9pbnQoKSArDQogIGdlb21fbGluZSgpICsNCiAgbGFicygiUHVibGlzaGluZyBGcmVxdWVuY3kgYnkgRGF5IiwgDQogICAgICAgc3VidGl0bGUgPSAiSmFudWFyeSwgMjAxNSIpDQogIA0KcHVibGlzaGVkX292ZXJfdGltZQ0KYGBgDQojIyBJbnRlcmFjdGl2ZQ0KDQpVc2luZyBQbG90dGx5J3MgYGdncGxvdGx5YCBmdW5jdGlvbiwgZ2VuZXJhdGUgdmlzdWFsaXphdGlvbnMgdGhhdCBhcmUgYXZhaWxhYmxlIGZvciBpbnRlcmFjdGl2ZSBtb3VzaW5nIChpLmUuIHN1YnNldHRpbmcgYW5kIGV4cGxvcmluZykuICBHYWRnZXRzIHN1Y2ggYXMgc2xpZGVycywgZHJvcC1kb3duIG1lbnVzLCBzZWxlY3Rpb24gYm94ZXMgYW5kIHJhZGlvIGJ1dHRvbnMgYXJlIGF2YWlsYWJsZSBhbmQgZXNwZWNpYWxseSB1c2VmdWwgd2hlbiBjb21iaW5pbmcgbGlicmFyeShjcm9zc3RhbGspIHdpdGggbGlicmFyeShmbGV4ZGFzaGJvYXJkcykgW2FzIHNlZW4gaW4gdGhlIG9wZW5pbmcgdGFiIG9mIHRoaXMgZGVtb25zdHJhdGlvbiBkYXNoYm9hcmRdKGh0dHBzOi8vcmZ1bi1mbGV4ZGFzaGJvYXJkcy5uZXRsaWZ5LmNvbS8pDQoNCg0KYGBge3J9DQpnZ3Bsb3RseShwdWJsaXNoZWRfbGFuZ3VhZ2VzX2JhcmdyYXBoKQ0KYGBgDQoNCg0KYGBge3J9DQpnZ3Bsb3RseShwdWJsaXNoZWRfb3Zlcl90aW1lKQ0KYGBgDQoNCg==